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'omputer  programs  may  be  regarded  as  formal  mathematical  objects  whose  properties  are  subject  to 
mathematical  proof.  Program  verification  is  the  use  of  formal,  mathematical  techniques  to  debug 
software  and  software  specifications^ 

1.  Code  Verification 

How  are  the  properties  of  computer  programs  proved?  We  discuss  three  approaches  in  this  article: 
inductive  invariants,  functional  semantics,  and  explicit  semantics.  Because  the  first  approach  has  received 
by  far  the  most  attention,  it  has  produced  the  most  impressive  results  to  date.  However,  the  field  is  now 
moving  away  from  the  inductive  invariant  approach. 


1.1.  Inductive  Assertions 

The  so-called  Floyd-Hoare  inductive  assertion  method  of  program  verification  [25,  33]  has  its  roots  in  the 
classic  Goldstine  and  von  Neumann  reports  [53]  and  handles  the  usual  kind  of  programming  language,  of 
which  FORTRAN  is  perhaps  the  best  example.  In  this  style  of  verification,  the  specifier  ‘annotates* 
certain  points  in  the  program  with  mathematical  assertions  that  are  supposed  to  describe  relations  that 
hold  between  the  program  variables  and  the  initial  input  values  each  time  ‘control*  reaches  the  annotated 
point.  Among  these  assertions  are  some  that  characterize  acceptable  input  and  the  desired  output.  By 
exploring  all  possible  paths  from  one  assertion  to  the  next  and  analyzing  the  effects  of  intervening 
program  statements  it  is  possible  to  reduce  the  correctness  of  the  program  to  the  problem  of  proving 
certain  derived  formulas  called  verification  conditions. 

Below  we  illustrate  the  idea  with  a  simple  program  for  computing  the  factorial  of  its  integer  input  N 


flowchart 


assertion 


The  first  assertion  is  the  ‘input  assertion*  and  might  be  something  like  0<£N.  When  proving  this 
program  correct  the  input  assertion  may  be  assumed  for  the  initial  value  of  N,  N0.  The  second  assertion 
is  the  ‘loop  invariant*  and  states  the  relations  that  bold  among  the  variables.  The  third  assertion  is  the 
‘output  assertion;*  in  this  program  it  is  A=N0!,  where  the  mathematical  definition  of  N!  is  provided 
axiomatic  ally: 

/ 
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N!  =  < 

I  1*01-1)!.  otherwise 

\ 

The  verification  conditions  for  this  problem  ut  fhowi  below. 

path  froa  input  to  loop: 

Input  ->  loopd.^j.^p 

path  tree  loop  to  loop: 

loopCA.N.V  *lW->  loopO^A.N-l.No) 


path  tram  loop  to  output: 
loopCA.N.!^)  h  N=0  ->  outpntCA.N^) 

It  is  claimed  that  if  these  three  formulas  are  theorems  then  whenever  the  program  b  started  on  a 
nonnegative  integer  N,  the  final  value  of  A  is  N!,  provided  the  program  terminates. 

The  transformation  of  a  properly  annotated  flowchart  into  verification  conditions  can  be  done 
mechanically.  Such  a  program  is  called  a  verification  condition  tenerator.  The  flowchart  is  usually 
presented  as  a  program  in  some  ordinary  programming  language.  Mechanical  verification  systems  based 
on  the  inductive  assertion  method  usually  consist  of  two  main  subsystems:  a  verification  condition 
generator  and  an  automatic  theorem-prover  or  proof  checker  to  prove  the  verification  conditions. 

The  first  mechanical  program  verification  system  was  developed  by  King  [36|,  a  student  of  Floyd's. 
Many  verification  systems  have  been  developed  since  [28,  35,  30,  20,  6|. 

Using  techniques  similar  to  the  generation  of  verification  conditions  it  is  possible  to  prove  termination 
and  absence  of  runtime  errors.  Consider  for  example  the  claim  made  for  the  system  described  in  [6]: 

If  a  FORTRAN  subprogram  is  accepted  and  proved  by  the  system  and  the  program  can  be 
loaded  onto  a  FORTRAN  processor  that  meets  the  ANSI  specification  of  FORTRAN  [52,  1] 
and  certain  parameteriied  constraints  on  the  accuracy  of  arithmetic,  then  any  invocation  of 
the  program  in  an  environment  satisfying  the  input  condition  of  the  program  will  terminate 
without  run-time  errors  and  will  produce  an  environment  satisfying  the  output  condition  of  the 
program. 

The  verification  conditions  generated  are  proved  by  the  Boyer-Moore  theorem  prover  |5).  Among  the 
FORTRAN  programs  proved  correct  mechanically  by  the  above  described  system  are  a  fast  string 
searching  algorithm  |6|,  an  integer  square  root  algorithm  using  Newton’s  method  (7),  and  a  linear  time 
majority  vote  algorithm  |9].  These  programs  are  each  relatively  small,  requiring  no  more  than  a  page  of 
code.  However,  the  correctness  arguments  are  fairly  deep. 

Two  of  the  most  widely  known  verification  systems,  the  Stanford  Verifier  by  David  Luckham  and  his 
students  at  Stanford  University  and  the  Gypsy  Verification  Environment  by  Don  Good  and  his  colleagues 
at  the  University  of  Texas  at  Austin,  have  been  used  to  verify  significantly  larger  programs.  Unlike  the 
FORTRAN  verifier  above,  these  two  verification  systems  present  the  user  with  an  integrated  set  of  tools 
and  specially  tailored  programming  languages  designed  to  make  verification  more  convenient. 

The  programming  language  supported  by  the  Stanford  Verifier  [35]  is  a  variant  of  PASCAL.  The 
theorem-prover  used  is  a  rewrite  rule  based  simplifier  built  on  a  decision  procedure  by  Oppen  and 
Nelson  |44).  The  most  significant  verification  task  accomplished  with  that  system  to  date  is  the 
verification  of  a  compiler  for  PASCAL,  by  Polak  [47].  The  total  amount  of  executable  code  verified  in 
that  application  is  around  3000  lines. 


The  Gypsy  Verification  Environment  (GVE)  supports  the  programming  language  Gypsy,  which  is  a 
derivative  of  Pascal  providing  a  somewhat  cleaner  semantics  and  concurrency  [20).  The  theorem-prover 
used  in  GVE  was  adapted  from  a  prover  by  Bledsoe  |4[.  GVE  was  used  to  verify  the  largest  program 
mechanically  verified  to  date:  a  communications  interface  to  a  computer  network  (48|.  The  interface 
consists  of  over  4200  lines  of  executable  code.  Some  2600  verification  conditions  were  proved 
mechanically  in  that  effort.  GVE  has  also  been  used  to  verify  a  'message  flow  modulator*  which,  in 
simple  terms,  is  a  switch  on  a  communications  line  that  shunts  to  a  branch  line  messages  with  certain 
properties. 

1.2.  Functional  Semantics 

Another  approach  to  program  verification  is  to  transform  the  flowchart  into  a  mathematical  function 
from  the  input  state  to  the  output  state.  This  approach  was  initially  suggested  by  McCarthy  [40], 

The  loop  in  the  flowchart  presented  above  can  be  transformed  into  the  following  recursive  function 
provided  the  input  is  known  to  be  a  natural  number  and  one  assumes  that  the  value  of  A  is  the  only 
interesting  component  of  the  final  state: 

/ 

I  A.  if  K=0 
loop  (N,  A)  =  < 

I  loop(N-l,N*A) ,  otherwise 

Since  the  loop  of  the  flowchart  is  entered  after  initializing  A  to  1,  the  expression  Ioop(N,l)  represents  the 
final  value  of  A  computed  by  the  program. 

This  transformation  process  can  be  carried  out  mechanically  by  a  program  that,  like  a  verification 
condition  generator,  knows  the  semantics  of  the  programming  language.  Once  a  program  has  been 
transformed  into  a  function  its  properties  may  be  proved  by  standard  mathematical  techniques.  For 
example,  the  specification  of  the  factorial  program  is  k>op(N,l)  ■*=  N!. 

This  theorem  is  easily  proved  by  first  proving,  by  induction,  the  more  general  theorem  k»op(N,A)  *■ 
A*N!. 

While  this  approach  can  in  principle  be  applied  to  any  programming  language  with  a  well  understood 
semantics,  it  is  in  fact  most  often  applied  to  applicative  programming  languages  where  the  initial  step 
—  the  transformation  of  a  program  into  a  function  —  is  unnecessary. 

The  Boyer- Moore  theorem-prover  has  been  used  to  prove  properties  of  many  recursive  functions. 
Among  the  theorems  proved  are  the  invertibility  of  the  Rivest,  Shamir,  and  Adleman  public  key 
encryption  algorithm  [11],  the  soundness  and  completeness  of  a  propositional  calculus  decision 
procedure  (5),  the  soundness  of  an  arithmetic  simplifier  now  used  in  the  system  [8|,  the  termination  of 
Takeuchi's  function  [42),  and  the  correctness  of  many  elementary  list  and  tree  processing  functions. 

Examples  of  other  work  in  this  area  include  [15,  17,  2].  The  Edinburgh  LCF  system  (discussed  below) 
has  also  been  used  to  prove  properties  of  recursive  functions;  among  the  theorems  proved  with  LCF  are 
the  correctness  of  a  parser  and  the  correctness  of  the  unification  algorithm  [19,  46). 
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1.3.  Explicit  Semantics 

In  both  the  inductive  assertion  sad  Inactions!  program  verification  methods,  the  semantics  of  a  voa 
Neumann  language  are  embedded  in  some  algorithm  such  as  a  verification  condition  generator.  An 
alternative  approach  is  to  make  programs  be  objects  in  the  logic  (typically  tree  structures)  and  then  to 
spell  out  the  semantics  of  programs  explicitly  with  axioms.  Scott-Strachey  denotations!  semantics  (50)  is 
an  example  of  this  approach  to  program  verification.  For  example,  the  Scott-Strachey  semantics  of  a 
programming  language  containing  an  assignment  statement  (var  exprj  is  likely  to  have  an  axiom  such 
as: 

evalCfvar  :=  expr;s]  ,a)  =  eval (s.bind (var,val(expr,a)  ,a3) 
which  asserts  that  the  meaning,  in  environment  a,  of  a  sequence  of  statements  beginning  with  the 
assignment  [var  expr),  is  the  meaning  of  the  tail  of  the  sequence  in  the  environment  obtained  from  a 
by  assigning  the  variable  var  the  value  of  expr  in  a.  In  this  setting,  a  program  verification  system  consists 
simply  of  a  mechanised  logic  together  with  the  axioms  for  defining  semantics. 

The  Edinburgh  LCF  system  (31)  is  a  mechanization  of  Scott’s  Logic  for  Computable  Functions  in  which 
one  can  define  the  semantics  of  programming  languages.  The  LCF  system  is  similar  to  a  proof  checker  in 
that  it  provides  low  level  primitives  for  manipulating  formulas  in  the  logic,  rather  than  a  sophisticated 
heuristic  search  strategy.  However,  LCF  also  provides  an  extremely  flexible  metalanguage  in  which  one 
can  build  (i.e.,  program)  proof  procedures  or  theorem-proven.  The  Edinburgh  LCF  has  been  used  to 
prove  the  equivalence  of  two  different  semantic  definitions  (18). 

Often  the  explicit  definition  of  the  semantics  takes  the  form  of  an  interpreter  for  the  programming 
language.  For  example,  in  (12,  10]  Boyer  and  Moore  define  an  interpreter  for  Pure  LISP  and  then  use 
their  mechanical  theorem-prover  to  prove  that  Pure  LISP  is  Turing  complete  and  that  the  halting  problem 
for  Pure  LISP  is  unsolvable. 

2.  Other  Program  Verification  Applications 

Thus  far  we  have  concentrated  on  the  verification  of  sequential  programs  with  respect  to  some 
specifications.  There  are  several  other  related  areas  of  program  verification  that  we  will  simply  mention 
here. 

e  The  mechanical  verification  of  the  properties  of  abstract  data  types.  See  (10,  43,  26,  27). 

•  The  mechanical  verification  of  concurrent  processes  or  networks  of  programs.  Among  the 
seminal  papers  devoted  to  theoretical  understanding  of  concurrency  are  those  by  Owicki  and 
Gries  (45],  Lamport  [38],  and  Hoare  |34|.  Mechanical  proofs  of  properties  of  concurrent 
systems,  in  addition  to  the  previously  cited  network  interface  proof  by  Good  with  the  Gypsy 
system,  include  the  verification  of  protocols  by  Divito  [23]  and  the  proof  of  the  correctness  of  a 
concurrent  sort  routine  by  Lengauer  (39] . 

#  The  mechanical  verification  of  properties  of  specifications.  Since  specifications  are  often 
simpler  to  reason  about  than  programs,  there  have  been  several  attempts  to  reason 
mechanically  about  specifications.  This  method  has  been  used  to  try  to  establish  the 
"security*  of  operating  system  designs.  One  such  checker  is  that  by  Feiertag  [24].  The  idea  of 
■design  verification'  was  also  used  in  the  attempt  to  establish  the  reliability  of  SRI’s  Software 
Implemented  Fault  Tolerant  (SIFT)  system  (49).  Of  course,  a  program  whose  design  has  been 
verified  is  unworthy  of  trust  until  the  running  program  has  been  shown  to  implement  the 
design.  Especially  to  be  distrusted  are  those  software  products  constructed  by  two  unrelated 
teams:  those  who  write  the  code  and  those  who  simultaneously  and  independently  write  the 
formal  specifications,  which  are  checked  for  security.  Alas,  several  such  projects  are  currently 
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funded  by  the  U.S.  government.  This  travesty  of  mathematical  proof  has  been  defended  by  a 
noted  logician  as  at  least  giving  the  government  better  documentation.  The  Department  of 
Defense  has  published  official  standards  authorising  this  nonsense. 


3.  Problems  and  Current  Directions 

Several  important  programming  areas  have  been  virtually  ignored  in  program  verification.  One  such 
area  is  the  the  mechanical  verification  of  real-time  control  programs.  A  minor  investigation  into  the  area 
was  done  by  Boyer  and  Moore  in  (13)  in  which  they  used  their  theorem-prover  to  prove  that  a  simple 
program  keeps  a  vehicle  aon  course*  in  a  varying  cross  wind.  A  major  problem  in  real-time  control 
verification  is  the  specification  of  the  non-digital  world  with  which  such  programs  interact.  A  related  area 
of  concern  is  hardware  verification,  where  timing  and  interrupt  handling  are  major  problems.  An  initial 
investigation  was  conducted  by  Wagner  [54],  who  used  Weyhrauch's  FOL  system  [56]  to  prove  properties 
of  circuits  for  such  basic  tasks  as  counting  and  multiplying.  Finally,  an  extremely  important  area  that  has 
received  almost  no  attention  is  the  mechanical  verification  of  floating  point  algorithms. 

Despite  the  successes  of  program  verification,  there  b  a  widespread  feeling  that  the  use  of  verification 
conditions  and  the  use  of  today’s  complicated  von  Neumann  programming  languages  are  major 
impediments  to  progress.  The  inductive  assertion  method  suffers  the  disadvantage  that  the  user’s  input 
—  a  program  and  its  specification  —  is  transformed  into  a  mass  of  formulas  which  bear  little  resemblance 
to  either.  When  one  of  these  formulas  fails  to  be  a  theorem  the  user  must  deduce  whether  the  bug  is  in 
the  program  or  the  specification  and  what  to  do  about  it.  This  is  a  task  similar  to  deducing  the  location 
of  a  black  body  by  observing  the  effect  of  its  gravitational  pull  on  visible  neighbors.  The  semantics  of 
today’s  programming  languages  further  complicates  the  problem. 

The  adoption  of  ’applicative*  or  "functional*  programming  languages  eliminates  the  first  problem 
because  proofs  can  be  conducted  in  the  language  in  which  the  program  is  written.  In  addition,  applicative 
languages  tend  to  be  simpler  than  von  Neumann  languages.  The  simplicity  of  program  verification  in  the 
setting  of  an  applicative  programming  language  has  contributed  to  the  growing  interest  in  such  languages. 
Among  the  applicative  languages  currently  being  developed  are  SASL  by  Tamer  [51],  the  reduction 
language  of  Backus  [3],  LISPKIT  by  Henderson  [32],  and  the  many  variants  of  predicate  calculus  as  a 
programming  language  (37|.  However,  applicative  languages  are  widely  considered  to  be  too  inefficient  for 
many  applications  and  much  of  the  work  in  program  verification  at  the  moment  is  actually  addressed  at 
improving  their  efficiency.  One  of  the  methods  used  is  to  exploit  their  simple  semantics  and  use 
automated  reasoning  to  deduce  the  correctness  of  optimisations  (21,  4l|. 

A  question  that  frequently  comes  up  is  "Have  you  verified  the  verifier?"  Perhaps  surprisingly,  this 
metamathematical  question  is  often  asked  by  engineers  (e.g.,  project  monitors  in  NASA  and  the  FAA)  not 
merely  by  pointy  headed  academics.  Of  course,  if  a  machine  ever  answers  the  question  *Do  you  ever  lie?" 
the  answer  will  be  no  more  informative  than  when  a  human  answers  the  question.  However,  several 
approaches  to  "verified  verifiers*  are  being  pursued,  ranging  from  running  the  output  of  the  theorem- 
prover  through  a  primitive  but  trusted  proof  checker  to  bootstrapping  from  such  a  proof  checker  to  a 
verified  automatic  theorem  prover  [22,  55,  14,  8,  31]. 

The  difficulties  of  verification  notwithstanding,  there  is  widespread  interest  in  the  field.  The  U.S. 
Government  has  already  issued  R.F.Q.’s  requiring  various  forms  of  mechanical  verification.  In  addition, 
the  recently  established  Department  of  Defense  Computer  Security  Center  has  defined  several  levels  of 
software  certification,  the  highest  level  being  mechanical  code  verification.  The  potential  market  for 
practical  program  verification  technology  can  be  glimpsed  by  considering  the  number  of  everyday 
products,  ranging  from  ballpoint  pens  to  automobiles,  containing  microchips  and  then  considering  the  coat 


of  recalling  those  products  due  to  bugs  in  their  software. 
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